Optical signal processing with illumination-encoded filters

Recently, computer vision researchers have shown that orthogonal functions and computational techniques from the signal processing framework can be mapped directly into the scene using projector-camera systems. These scene-space signal processing algorithms are achieved with illumination-encoded functions as primitives and computations derived from surface reflection models. Some examples of this new optical approach include convolution filtering and aliasing-canceling filter banks. In this paper we present computational techniques for realizing fundamental elements of the signal processing framework in the 3D scene domain. The motivation for optical computation directly in the scene is to avoid information loss when the rich 3D scene is reduced to an image. The computations are at subpixel resolution because they are performed within each camera sensor. Scene-space filtering applies 2D operators to locally planar 3D surfaces based on the optical coupling of the scene surface topology with projector-camera devices. Signal processing issues such as sampling geometry, dynamic range, mathematical operators, and resolution are addressed. We report a novel subpixel point correspondence technique for accurate camera sensor footprint localization in general scenes. It is a parallelizable optical clipping algorithm related to the polygon clipping and box filter anti-aliasing algorithms used in computer graphics. It replaces the regular coordinates of an image with a Email addresses: [email protected] (Yves D. Jean), [email protected] (Yves D. Jean) URL: www.lehman.edu/faculty/yves (Yves D. Jean), www.lehman.edu/faculty/yves (Yves D. Jean) Preprint submitted to Computer Vision and Image Understanding July 2, 2010 local surface parametrization in projector coordinates. The result is subpixel resolution filter responses. We provide experiments and results to evaluate the performance of our scene-space filtering techniques with both planar and non-planar objects.